SurveyFail
At the end of August 2009, two researchers, Oli Ogas and Sai Gaddam, posted a survey of Fan fiction writers and publicised it widely on LiveJournal. The survey contained many problematic questions and led to extensive discussion by the fannish community, and eventually complete repudiation and mockery of the researchers, their goals, and methods. The survey and its aftermath raised issues of Essentialism, Social expectations, Condescension and more. Origins Olas and Gaddam recently signed a contract for a book called "Rule 34: What Netporn Tells Us About the Brain". This research is intended for that book. The researchers contacted shaggirl, the moderator of the crack_van community on LJ, and asked for her help in creating and publicising the survey. Numerous LJ threads occurred at this point, in which shaggirl and others tried to explain fandom to the researchers. * Original contact with shaggirl * Shaggirl's first response * mecurtin's offers stats on fanfic participation * mecurtin on fanfic archives post-2004 As it became apparent that the survey was not going to be understanding of fandom, people started posting to tell people not to take it. Eg. * http://eruthros.dreamwidth.org/273840.html * http://tencrush.livejournal.com/208332.html The survey and its questions The survey was posted on (date) and publicised widely on LJ, including on the crack_van community. The survey asked questions involving: * Gender, race, age, and sexual preference of participants * Types of fanfic stories they read and wrote * Which fictional characters and scenarios they "identified" with * Questions about drug and alcohol use, masturbation, etc. * Intelligence (based on SAT scores) * Memories of early life events A full transcript of the questions is available. Also here. The phrasing of many of the questions was problematic. Fans took the writers to task over a range of issues, including: * Terminology used for gender, race, and sexual identity * US-centrism of many questions * Failing to understand fannish culture and fanfic practice * Assumptions about respondents' interest in fandom and fanfic * Questions that did not represent a full range of options, and had no "other"/write-in option available People expressed their concern on Ogi Ogas's LJ, which has since been locked. However, other discussions of the survey occurred in the following places: * http://kyuuketsukirui.livejournal.com/1115452.html : "It's a badly-designed survey. It's a heavily-biased survey. Even if you agree with their mission statement, you shouldn't take the survey because it's a piece of shit survey. Any survey that requires you to lie to proceed is a bad survey (and since there is nothing that says you can skip a question and still proceed, many people will indeed lie in order to be able to move on)." * Satire of the survey Others took issue with the underlying purpose of the research. Eg. * http://kindkit.livejournal.com/396740.html : "Translated out of sociobiologist into English, I believe this says: We're out to prove what everyone with any common sense already knows, namely that men and women are completely different! Men are from Mars, women are from Venus! Men (not counting queers, of course) want to fuck every woman in sight and are turned on by teenage girls with large breasts; women (not counting queers, of course) want to tie a man down monogamy and are turned on by large bank accounts! And all of this is completely hardwired, inherited from our distant ancestors, and in no way a product of economic, political, and social structures!" Privacy and ethics concerns Although the survey claimed to be private and confidential, there is evidence that they were collecting IP addresses and were able to associate them with individuals on LJ (where they also tracked IPs). * http://jonquil.dreamwidth.org/864306.html Many people also pointed out that the survey was open to minors, and asked questions about sexual activity and drug use, and that no IRB would allow such a survey. It rapidly became apparent that the researchers had not been through a suitable process of ethical review. A number of people tried to contact Boston University and/or the IRB there. * Rivkat doesn't think IRBs are the right tool for the job An Australian PhD candidate is considering using the whole incident as a teaching example of how not to conduct research: I am always warning my students when I teach research methods that something like this can happen - that this is why I’m so harsh on their research designs. Welcome to my new case study. I’m serious. I’m thinking of assigning parts of this trainwreck when I teach research methods next term. * Wearing the Juice: A Case Study in Research Implosion Researchers' relationship with Boston University * Boston University denies affiliation Non-apology Ogas and Gaddam then issued an alleged apology that continued their Othering and general dismissal of fandom's concerns. The following two posts do a good job of dissecting the "apology": * Ogi apologises by insulting fandom some MOAR (by pine on DW) * How not to apologize (by piranha on DW) Issues in context * daedala discusses the survey in the context of cultural appropriation Broader coverage * MetaFilter Category:Incidents Category:2009